Several formats have been presented for storing pixel data in video subsystems. One approach is providing twenty-four bits of red, green, blue (RGB) information per pixel. This approach yields the maximum color space required for video at the cost of three bytes per pixel. Depending on the number of pixels in the video subsystem, the copy/scale operation could be over-burdened by this.
A second approach is a compromise with the twenty-four bit system. This approach is based on sixteen bits of RGB information per pixel. Systems of this nature require fewer bytes for the copy/scale operation but have the disadvantage of less color depth. Additionally, since the intensity and color information are encoded in the R, G and B components of the pixel, this approach does not take advantage sensitivity of the human eye to intensity and its insensitivity to color saturation. Other sixteen bit systems have also been proposed in which the pixels are encoded in a YUV format such as 6, 5, 5 and 8, 4, 4. Although these systems are somewhat better than the sixteen bit RGB approach, the sixteen bit YUV format does not performance as well as twenty bit systems.
Eight bit color lookup tables provide a third approach to this problem. The color lookup table method uses eight bits per pixel as an index into a color map that typically has twenty bits of color space. This approach has the advantages of low byte count while providing twenty bit color space. However, there are only two hundred fifty-six colors available on the screen in this approach and image quality may be somewhat poor.
Dithering techniques that use adjacent pixels to provide additional colors have been demonstrated to have excellent image quality even for still images. However, these dithering techniques often require complicated algorithms and specialized palette entries in a digital-to-analog converter as well as almost exclusive use of a color lookup table. The overhead of running the dithering algorithm must be added to the copy/scale operation.
Motion video in some prior art systems is displayed in a 4:1:1 format referred to as the nine bit format. The 4:1:1 notation indicates that there are four Y samples horizontally for each UV sample and four Y samples vertically for each UV sample. If each sample is eight bits then a four by four block of pixels uses eighteen bytes of information or nine bits per pixel. Although image quality is good for motion video the nine bit format may be unacceptable for the display of high quality stills. In addition, the nine bit format does not integrate well with graphics subsystems. Other variations of the YUV subsampled approach include an eight bit format.
Systems integrating a graphics subsystem display buffer with a video subsystem display buffer generally fall into two categories. The two types of approaches are known as: (1) single active frame buffer architecture and (2) dual frame buffer architecture. The single active frame buffer architecture is the most straight forward approach and consists of a single graphics controller, a single digital-to-analog converter and a single frame buffer. In its simplest form, the single active frame buffer architecture represents each pixel on the display using bits in a display buffer which are consistent in their format regardless of the meaning of pixel on the display.
Thus, graphics pixels and video pixels are indistinguishable in the memory of the frame buffer. However, the single active frame buffer architecture graphics/video system, or the single active frame buffer architecture visual system, does not address the requirements of the video subsystem very well. Full screen motion video on the single active frame buffer architecture visual system requires updating every pixel in the display buffer thirty times a second.
In a typical system the display may be on the order of 1280 bits by one kilobyte by eight bits, Even without the burden of writing over thirty megabytes per second to the display buffer, eight bit video by itself does not provide the require video quality. Thus the single active frame buffer architecture system may either expand to sixteen bits per pixel or implement the eight bit YUV subsampled technique. Since sixteen bits per pixel yields over sixty megabytes per second into the frame buffer, it is an unacceptable alternative. A further disadvantage of this single frame buffer architecture is the need for redundant frame memory. This is caused by the need to store both a graphics pixel and a video pixel for at least a portion of the display.
The second category of architecture which integrates video and graphics is the dual frame buffer architecture. The dual frame buffer architecture visual system involves mixing two otherwise free-standing single frame buffer systems at the analog back end with a high-speed analog switch. Since the video and graphics subsystems are both single frame buffer designs each one can make the necessary tradeoffs in spatial resolution and pixel depth almost independently of the other subsystem. Dual frame buffer architecture visual systems also include the feature of being loosely-coupled. Since the only connection of the two subsystems is in the final output stage, the two subsystems may be on different buses within the system. The fact that the dual frame buffer architecture video subsystem is loosely-coupled to the graphics subsystem is usually the major reason such systems, which have significant disadvantages, are typically employed.
Dual frame buffer architecture designs typically operate in a mode that has the video subsystem genlocked to the graphics subsystem. Genlocking requires that both subsystems start to display their first pixel at the same time. If both subsystems run at the same horizontal line frequency with the same number of lines, then mixing of the two separate pixel streams may be performed with predictable results.
Since both pixel streams run at the same time, the process may be thought of as having video pixels underlaying the graphics pixels. If a determination is made not to show a graphics pixel, then the video information underlaying it shows through. In dual frame buffer architecture designs, it is not necessary for the two subsystems to have the same number of horizontal pixels. As an example, some known systems may have three hundred fifty-two video pixels underneath one thousand twenty-four graphics pixels.
The decision whether to show the video information or graphics information at each pixel position in dual frame buffer architecture visual systems is typically made on a pixel by pixel basis in the graphics subsystem. A technique often used is chroma keying. Chroma keying involves detecting a predetermined color in the graphics digital pixel stream or a predetermined color entry in a color lookup table and selecting either graphics or video accordingly. Another approach detects black in the graphics analog pixel stream because black is the easiest graphics level to detect. This approach is referred to as black detect. In either case, keying information is used to control the high speed analog switch and the task of integrating video and graphics on the display is reduced to painting the keying color in the graphics display wherever video pixels are to be displayed.
There are several disadvantages to dual frame buffer architecture visual systems. The goal of high integration is often complicated by the requirement that there be two separate, free standing subsystems. The cost of having duplicate digital-to-analog converters, display buffers, and cathode ray tube controllers may be significant. The difficulty of genlocking the pixel streams and the cost of the high-speed analog switch are two more disadvantages. In addition, placing the analog switch in the graphics path has detrimental effects on the quality of the graphics display. This becomes a greater problem as the spatial resolution and/or line rate of the graphics subsystem increases. A further disadvantage of the dual frame buffer architecture is the same as that found in the single active frame buffer architecture, the need for redundant frame memory. This is caused by the need to store both a graphics pixel and a video pixel for at least a fraction of the display. For both the single active frame buffer and the dual frame buffer the two pixels are sent to either-a digital multiplexor or an analog multiplexor and a decision is made on which is displayed.
Digital-to-analog converters within these visual frame buffer architectures are important high performance components. The digital-to-analog converters of these architectures may accept YUV color information and the RGB color information simultaneously to provide chroma keying according to the received color information. In prior art chroma keying systems a decision is made for each pixel of a visual display whether to display a pixel representative of the YUV color value or a pixel representative of the RGB color value. The RGB value within a chroma keying system is typically provided by the graphic subsystem. The YUV value within a chroma keying system is typically provided by a video subsystem. Because the digital-to-analog converters required to select between pixels are such high performance devices the use of two of them rather than one adds a significant cost to a system.
In many of these conventional chroma keying systems the determination regarding which pixel is displayed is based upon the RGB color value and in a single display image there may be a mixture of pixels including both YUV pixels and RGB pixels. Thus it will be understood that each pixel displayed using conventional chroma keying systems is either entirely a video pixel or entirely a graphics pixel. Chroma keying merely determines which to select and provides for the display of one or the other.
"Visual Frame Buffer Architecture", U.S. patent application Ser. No. 870,564, filed by Lippincott, and incorporated by reference herein, teaches a color lookup table method which addresses many of the problems of prior art systems. In the Lippincott method an apparatus for processing visual data is provided with storage for storing a bit plane of visual data in a one format which may, for example, be RGB. A graphics controller is coupled to the storage by a data bus, and a graphics controller and the storage are coupled through a storage bus. Further storage is provided for a second bit plane of visual data in another format different from the first format. The second format may, for example, be YUV. The further storage is coupled to the graphics controller by a data bus. The second storage is also coupled to the graphics controller through the storage bus.
The method taught by Lippincott merges a pixel stream of visual data stored on the first storage and visual data stored on the further storage using only a single digital-to-analog converter. The merged pixel stream is then displayed. A disadvantage of this type of frame buffer architecture is the need for redundant frame memory. This is caused by the need to store both a graphics pixel and a video pixel for at least a fraction of the display.